Neural network-based high-resolution image restoration method and system

ABSTRACT

The present invention relates to a neural network-based high-resolution image restoration method and system, including: performing feature extraction on a target frame in a network input to obtain a first feature, performing feature extraction on a first frame and an adjacent frame and an optical flow between the first frame and the adjacent frame to obtain a second feature, and concatenating the first feature and the second feature to obtain a shallow layer feature; performing feature extraction and refinement on the shallow layer feature to obtain a plurality of output first features and a plurality of output second features; performing feature decoding on the plurality of output second features, and concatenating decoded features along channel dimensionality to obtain features; and performing weight distribution on the features to obtain final features, and restoring an image. The present invention can effectively help to improve image quality.

FIELD OF THE INVENTION

The present invention relates to the field of image restorationtechnologies, and more particularly to a neural network-basedhigh-resolution image restoration method and system.

DESCRIPTION OF THE RELATED ART

In modern life, the need for high-resolution images is very common, andhigh-resolution images or videos are required in a number of fields suchas security surveillance, medical imaging, target detection, and facerecognition. In a high-resolution image restoration technique, acorresponding high-resolution image is generated by using alow-resolution image as an input without other information, and thetechnique receives positive subjective and objective evaluation. Forexample, in an interpolation algorithm in a conventional method, theresolution of an image can be effectively improved by interpolationoperation. However, the interpolation algorithm receives rather negativesubjective and objective evaluation.

SUMMARY OF THE INVENTION

For this, a technical problem to be resolved by the present invention isto overcome the problem of rather negative subjective and objectiveevaluation because a high-resolution image restoration method in theprior art is complex, to provide a neural network-based high-resolutionimage restoration method and system that are simple and can improve thequality of a low-resolution image.

To solve the foregoing technical problems, the present inventionprovides a neural network-based high-resolution image restorationmethod, and the method includes: performing feature extraction on atarget frame in a network input to obtain a first feature, performingfeature extraction on a first frame and an adjacent frame of the firstframe and an optical flow between the first frame and the adjacent frameto obtain a second feature, and concatenating the first feature and thesecond feature to obtain a shallow layer feature; performing featureextraction and refinement on the shallow layer feature by using aniterative up and down sampling method to obtain a plurality of outputfirst features and a plurality of output second features; performingfeature decoding on the plurality of output second features, andconcatenating decoded features along channel dimension to obtainfeatures after a plurality concatenation; and performing weightdistribution on the features after the plurality of concatenation toobtain final features, and restoring an image by using the finalfeatures.

In an embodiment of the present invention, during feature extraction ofthe target frame in the network input, feature extraction is performedon the target frame by using one or two convolutional layers to obtainthe first feature; and during feature extraction of the first frame andan adjacent frame of the first frame and the optical flow between thefirst frame and the adjacent frame, feature extraction is performed onthe first frame, a dense optical flow between the first frame and asecond frame, and the second frame in a low-resolution image sequence byusing one or two convolutional layers.

In an embodiment of the present invention, a determination method ofperforming feature extraction and refinement on the shallow layerfeature by using an iterative up and down sampling method comprises:determining whether there are uncalculated adjacent frames in theshallow layer feature, where if yes, feature concatenation is performedon one obtained output first feature and a feature of an optical flowbetween a next frame and an adjacent frame of the next frame for use asan input for next iteration, and cyclic iteration is performed until allinput frames have been calculated; or if not, the process enters StepS3.

In an embodiment of the present invention, during the iterative up anddown sampling, a process of a single iteration up and down samplingincludes: a first convolutional layer, a first deconvolutional layer, asecond convolutional layer, a second deconvolutional layer, a thirdconvolutional layer, and a third deconvolutional layer.

In an embodiment of the present invention, the first convolutional layerand the first deconvolutional layer use the same convolution kernel,step size, and channel quantity; the second convolutional layer and thesecond deconvolutional layer use the same convolution kernel, step size,and channel quantity; and the third convolutional layer and the thirddeconvolutional layer use the same convolution kernel, step size, andchannel quantity.

In an embodiment of the present invention, an input of the firstconvolutional layer is the shallow layer feature, an input of the firstdeconvolutional layer is a result of the first convolutional layer, aninput of the second convolutional layer is a difference between a resultof the first deconvolutional layer and the shallow layer feature, aninput of the second deconvolutional layer is a result of the secondconvolutional layer, an input of the third convolutional layer is aresult of the second deconvolutional layer, and an input of the thirddeconvolutional layer is a difference between a result of the thirdconvolutional layer and the result of the second deconvolutional layer.

In an embodiment of the present invention, the number of iterative upand down sampling process is adjusted according to a requirement of anetwork scale.

In an embodiment of the present invention, during the iterative up anddown sampling, an output second feature obtained from each iteration issaved.

In an embodiment of the present invention, during restoration of theimage by using the final features, one or two convolutional layers areused.

The present invention further provides a neural network-basedhigh-resolution image restoration system, including: a featureextraction module, configured to: perform feature extraction on a targetframe in a network input to obtain a first feature, perform featureextraction on a first frame and an adjacent frame of the first frame andan optical flow between the first frame and the adjacent frame of thefirst frame to obtain a second feature, and concatenate the firstfeature and the second feature to obtain a shallow layer feature; anencoding and decoding module, configured to perform feature extractionand refinement on the shallow layer feature by using an iterative up anddown sampling method to obtain a plurality of output first features anda plurality of output second features; an encoding module, configuredto: perform feature decoding on the plurality of output second features,and concatenate decoded features along channel dimensionality to obtainfeatures after a plurality of concatenation; and a weight distributionmodule and a restoration module, configured to: perform weightdistribution on the features after the plurality of concatenation toobtain final features, and restore an image by using the final features.

Compared with the prior art, the foregoing technical solutions of thepresent invention has the following advantages:

For the neural network-based high-resolution image restoration methodand system of the present invention, feature extraction is performed ona target frame in a network input to obtain a first feature, and featureextraction is performed on a first frame and an adjacent frame of thefirst frame and an optical flow between the first frame and an adjacentframe to obtain a second feature, it not only implement initial featurefitting, but also adjust the scale of a network, so that the size of thenetwork parameter scan be controlled. The first feature and the secondfeature are concatenated to obtain a shallow layer feature, tofacilitate feature extraction and refinement of features. Featureextraction and refinement are performed on the shallow layer feature byusing an iterative up and down sampling method to obtain a plurality ofoutput first features and a plurality of output second features, so thatkey features of an inputted image can be adequately kept, to avoid avanishing gradient case during training. Feature decoding is performedon the plurality of output second features, and decoded features areconcatenated along channel dimensionality to obtain features after aplurality of concatenation. Weight distribution is performed on thefeatures after the plurality of concatenation to obtain final features,and an image is restored by using the final features. Because differentframes have different distances from the target frame and contributedifferently to reconstructed information, the step can effectively helpto improve image quality.

BRIEF DESCRIPTION OF THE DRAWINGS

To make the content of the present invention clearer and morecomprehensible, the present invention is further described in detailbelow according to specific embodiments of the present invention and theaccompanying draws. Where:

FIG. 1 is a flowchart of a neural network-based high-resolution imagerestoration method according to the present invention;

FIG. 2 is a schematic diagram of a neural network-based high-resolutionimage restoration system according to the present invention; and

FIG. 3 is a schematic diagram of a process of a single iteration up anddown sampling according to the present invention.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Embodiment 1

As shown in FIG. 1 , this embodiment provides a neural network-basedhigh-resolution image restoration method, including: Step S1: performingfeature extraction on a target frame in a network input to obtain afirst feature, performing feature extraction on a first frame and anadjacent frame of the first frame and an optical flow between the firstframe and the adjacent frame to obtain a second feature, andconcatenating the first feature and the second feature to obtain ashallow layer feature; Step S2: performing feature extraction andrefinement on the shallow layer feature by using an iterative up anddown sampling method to obtain a plurality of output first features anda plurality of output second features; Step S3: performing featuredecoding on the plurality of output second features, and concatenatingdecoded features along channel dimension to obtain features after aplurality of concatenation; and Step S4: performing weight distributionon the features after the plurality of concatenation to obtain finalfeatures, and restoring an image by using the final features.

For the neural network-based high-resolution image restoration method inthis embodiment, in Step S1, feature extraction is performed on a targetframe in a network input to obtain a first feature, and featureextraction is performed on a first frame and an adjacent frame of thefirst frame and an optical flow between the first frame and the adjacentframe to obtain a second feature, it not only implement initial featurefitting, but also adjust the scale of a network, so that the size of thenetwork parameters can be controlled. The first feature and the secondfeature are concatenated to obtain a shallow layer feature, tofacilitate feature extraction and refinement of features. In Step S2,feature extraction and refinement are performed on the shallow layerfeature by using an iterative up and down sampling method to obtain aplurality of output first features and a plurality of output secondfeatures, so that key features of an inputted image can be adequatelykept, to avoid a vanishing gradient case during training. In Step S3,feature decoding is performed on the plurality of output secondfeatures, and decoded features are concatenated along channeldimensionality to obtain features after a plurality of concatenation. InStep S4, weight distribution is performed on the features after theplurality of concatenation to obtain final features, and an image isrestored by using the final features. Because different frames havedifferent distances from the target frame and contribute differently toreconstructed information, the step can effectively help to improveimage quality.

In Step S1, during feature extraction of the target frame in the networkinput, shallow layer feature extraction is performed on the target frameby using one or two convolutional layers to obtain the first feature;and during feature extraction of the first frame and an adjacent frameof the first frame and the optical flow between the first frame and theadjacent frame, feature extraction is performed on the first frame, adense optical flow between the first frame and a second frame, and thesecond frame in a low-resolution image sequence by using one or twoconvolutional layers.

As shown in FIG. 2 , shallow layer feature extraction is respectivelyperformed on a target frame L_(t) and a first frame and a dense opticalflow between the first frame and a second frame and the second frame[L_(t−2), F_(t−2), L_(t−1)] in a low-resolution image sequence by usingone or two convolutional layers, to respectively obtain a first featureI_(t) and a second feature f_(t−2).

The first feature I_(t) and the second feature f_(t−2) are concatenatedto obtain a shallow layer feature F_(conv(i),i=1). (i denotes a quantityof iteration).

The target frame is a middle frame in the network input. In thisembodiment, five frames are used as an example, and the middle frame isused as the target frame.

In Step S2, a determination method of performing feature extraction andrefinement on the shallow layer feature by using an iterative up anddown sampling method comprising: determining whether there areuncalculated adjacent frames in the shallow layer feature, where if yes,feature concatenation is performed on one obtained output first featureand a next frame and a feature of an optical flow between the next frameand an adjacent frame of the next frame for use as an input for a nextiteration, and cyclic iteration is performed until all input frames havebeen calculated; or if not, the process enters Step S4.

It is determined whether there are uncalculated adjacent frames in theshallow layer feature. If yes, feature concatenation is performed on oneobtained output first feature and a next frame and a feature of anoptical flow between the next frame and an adjacent frame of the nextframe. Let i=i+1. Feature extraction is performed again on theconcatenated feature to obtain a new feature. The new feature is used asan input to continue to perform iterative up and down sampling again toextract a feature. The cycle is repeated until all input frames havebeen calculated.

The iterative up and down sampling method includes a plurality of groupsof upsampling modules and downsampling modules, and cross layerconnections are used.

As shown in FIG. 3 , during the iterative up and down sampling process,a process of a single iteration up and down sampling includes: a firstconvolutional layer, a first deconvolutional layer, a secondconvolutional layer, a second deconvolutional layer, a thirdconvolutional layer, and a third deconvolutional layer.

the first convolutional layer and the first deconvolutional layer usethe same convolution kernel, step size, and channel quantity; the secondconvolutional layer and the second deconvolutional layer use the sameconvolution kernel, step size, and channel quantity; and the thirdconvolutional layer and the third deconvolutional layer use the sameconvolution kernel, step size, and channel quantity.

An input of the first convolutional layer is the shallow layer feature,an input of the first deconvolutional layer is a result of the firstconvolutional layer, an input of the second convolutional layer is adifference between a result of the first deconvolutional layer and theshallow layer feature, an input of the second deconvolutional layer is aresult of the second convolutional layer, an input of the thirdconvolutional layer is a result of the second deconvolutional layer, andan input of the third deconvolutional layer is a difference between aresult of the third convolutional layer and the result of the seconddeconvolutional layer.

The number of iterative up and down sampling process is adjustedaccording to a requirement of a network scale.

In Step S3, feature decoding is performed on the plurality of outputsecond features F_(iter)(i), and decoded features and decoded featuresin a previous cycle F_(iter) (i) are concatenated along channeldimensionality to finally obtain features F_(rec) after a plurality ofconcatenation.

In Step S4, because different frames have different distances from thetarget frame and contribute differently to reconstructed information,weight redistribution is performed on F_(rec) to obtain rec Duringrestoration of the image by using the final features, one or twoconvolutional layers are used. F_(rrec) is used to continue to performimage restoration to obtain a final target frame.

In the present application, during processing of adjacent frames and anoptical flow between the adjacent frames, dense optical flow extractionis first performed on inputted consecutive frames of image. It isassumed that the inputted consecutive frames of image are [L_(t−(k−1)/2). . . L_(t−2), L_(t−1), L_(t), L_(t+1), L_(t+2), L_(t+(k−1)/2)], where aquantity of the frames is k(k=2i+1,i=1,2,3 . . . ).

Data after dense optical flows are extracted is:

[L _(t−(k−1)/2) . . . L _(t−2) ,F _(t−2) ,L _(t−1) ,F _(t−1) ,L _(t) ,F_(t+1) ,L _(t+1) ,F _(t+2) ,L _(t+2) . . . L _(t+(k−1)/2)],

where L_(t) denotes a low-resolution frame, and F_(t) denotes a denseoptical flow between adjacent frames. Generally, during processing of aninput of a plurality of consecutive frames, a recurrent neural network(RNN) is often used. Therefore, the RNN can adequately extract temporaland spatial information of adjacent frames of image, so that sequenceinformation can be adequately processed. In the present invention, anRNN structure is not used. However, a characteristic of processingsequence information is kept.

Embodiment 2

This embodiment provides a neural network-based high-resolution imagerestoration system. An image is restored based on the neuralnetwork-based high-resolution image restoration method in Embodiment 1,and the principle of the system is the same as that of the method.

The neural network-based high-resolution image restoration systemincludes: a feature extraction module, configured to: perform featureextraction on a target frame in a network input to obtain a firstfeature, perform feature extraction on a first frame and an adjacentframe of the first frame and an optical flow between the first frame andthe adjacent frame to obtain a second feature, and concatenate the firstfeature and the second feature to obtain a shallow layer feature; anencoding and decoding module, configured to perform feature extractionand refinement on the shallow layer feature by using an iterative up anddown sampling method to obtain a plurality of output first features anda plurality of output second features; an encoding module, configuredto: perform feature decoding on the plurality of output second features,and concatenate decoded features along channel dimensionality to obtainfeatures after a plurality of concatenation; and a weight distributionmodule and a restoration module, configured to: perform weightdistribution on the features after the plurality of concatenation toobtain final features, and restore an image by using the final features.

The feature extraction module is configured to: perform initial featurefiltering on inputted low-resolution images, and extract a relativelysmall number of features to prepare for encoding. A convolutional layeris used to complete the part. Through feature extraction, initialfeature fitting can be implemented, and in addition the scale of anetwork can be adjusted, so that a parameter amount of the network canbe controlled.

The encoding and decoding module, that is, an iterative up and downsampling module, mainly uses the structure of a convolutional layer, adeconvolutional layer, and a residual network, and the residual networkhas adequate transferability, so that key features of an inputted imagecan be adequately kept, to avoid a vanishing gradient case duringtraining. Each encoding and decoding module includes an additional inputand previously trained results, so that time domain information can befully utilized, and information of each frame is fully utilized. This isused as additional information, which helps to restore a high-resolutioncurrent frame of image. The encoding and decoding module first scales upan image, then scales down the image, and then scales up the image, andup and down iteration is continuously performed, to better learn networkparameters. This process is a feedback process of the network. Differentfrom most network structures with only feedforward, feedback can betterextract image features.

As shown in FIG. 2 , specifically, a process of iterative up and downsampling includes six convolutional layers. The process includes: aconvolutional layer 1, a deconvolutional layer 1, a convolutional layer2, a deconvolutional layer 2, a convolutional layer 3, and adeconvolutional layer 3. Each convolutional layer and eachdeconvolutional layer use the same convolution kernel, step size, andchannel quantity. An input of the convolutional layer 1 is the shallowlayer feature. An input of the deconvolutional layer 1 is a result ofthe convolutional layer 1. An input of the convolutional layer 2 is adifference between a result of the deconvolutional layer 1 and theshallow layer feature. An input of the deconvolutional layer 2 is aresult of the convolutional layer 2. An input of the convolutional layer3 is a result of the deconvolutional layer 2. An input of thedeconvolutional layer 3 is a difference between a result of theconvolutional layer 3 and the result of the deconvolutional layer 2.

The process of encoding and decoding may include two or three times ofiteration. A specific quantity may be adjusted according to arequirement of a network scale, that is, an operation time.

In the encoding module, corresponding to each encoding and decodingmodule, the structure of each encoding and decoding module needs tocontribute to the final reconstruction, thereby providing validinformation. Therefore, each encoding and decoding training module isaccompanied with a further decoding module, and the module is formed bydeconvolutional layers, thereby fully extracting information obtainedthrough training of the encoding and decoding module.

The weight distribution module and the restoration module, finalfeatures F_(rec) extracted by the network are obtained after decodingand concatenating. The weight distribution module performs weightredistribution on previously obtained F_(rec) to obtain F_(rrec), sothat interference information such as artifacts can be adequatelyeliminated. The restoration module is completed by using adeconvolutional layer, and a final restored image is obtained by usingF_(rrec).

A main scenario of the present invention is a case of shooting picturesfrom a long distance. If the distance is excessively long, a pixel areaoccupied by a target tends to be relatively small, and problems such aslight and lens shake are all likely to cause blur in images. A fewframes in a lens are obtained through sampling within a short period oftime as an actual input to the network, to obtain a high-resolutionimage output. During use, a high-resolution image can be obtained veryconveniently by using a plurality of consecutive images as an input tothe network. During the training of the network, a plurality oflow-resolution images and a single high-resolution image are used asimage pairs, that is, Ht—— [L_(t−(k−1)/2) . . . L_(t−2), L_(t−1), L_(t),L_(t+1), L_(t+2) . . . L_(t+(k−1)/2)]. The plurality of low-resolutionimages and dense optical flows between the plurality of low-resolutionimages are used as inputs to the network, so that it is obtained that anoutput of the network is H_(t). S_(t) and H_(t) form a loss of networktraining. An objective of the network is to reduce a loss valuedominated by a loss, to update parameters, thereby obtaining a betterresult. The network uses an optimizer Adam, and an initial learning rateis set to 0.0001.

A person skilled in the art should understand that the embodiments ofthe present application may be provided as a method, a system or acomputer program product. Therefore, the present application may use aform of hardware only embodiments, software only embodiments, orembodiments with a combination of software and hardware. Moreover, thepresent application may use a form of a computer program product that isimplemented on one or more computer-usable storage media (including, butnot limited to a disk memory, a compact disc read-only memory (CD-ROM),an optical memory, and the like) that include computer usable programcode.

The present application is described with reference to the flowchartsand/or block diagrams of a method, a device (system), and the computerprogram product according to the embodiments of the present application.It should be understood that computer program instructions may be usedto implement each process and/or each block in the flowcharts and/or theblock diagrams and a combination of a process and/or a block in theflowcharts and/or the block diagrams. These computer programinstructions may be provided for a general-purpose computer, a dedicatedcomputer, an embedded processor, or a processor of any otherprogrammable data processing device to generate a machine, so that theinstructions executed by a computer or a processor of any otherprogrammable data processing device generate an apparatus forimplementing a specific function in one or more processes in theflowcharts and/or in one or more blocks in the block diagrams.

These computer program instructions may be stored in a computer readablememory that can instruct the computer or any other programmable dataprocessing device to work in a specific manner, so that the instructionsstored in the computer readable memory generate an artifact thatincludes an instruction apparatus. The instruction apparatus implementsa specific function in one or more processes in the flowcharts and/or inone or more blocks in the block diagrams.

These computer program instructions may be loaded onto a computer oranother programmable data processing device, so that a series ofoperations and steps are performed on the computer or the anotherprogrammable device, thereby generating computer-implemented processing.Therefore, the instructions executed on the computer or the anotherprogrammable device provide steps for implementing a specific functionin one or more processes in the flowcharts and/or in one or more blocksin the block diagrams.

Obviously, the foregoing embodiments are merely examples for cleardescription, rather than a limitation to implementations. For a personof ordinary skill in the art, other changes or variations in differentforms may also be made based on the foregoing description. Allimplementations cannot and do not need to be exhaustively listed herein.Obvious changes or variations that are derived there from still fallwithin the protection scope of the invention of the present invention.

1. A neural network-based high-resolution image restoration method,comprising steps of: Step S1: performing feature extraction on a targetframe in a network input to obtain a first feature, performing featureextraction on a first frame and an adjacent frame of the first frame andan optical flow between the first frame and the adjacent frame to obtaina second feature, and concatenating the first feature and the secondfeature to obtain a shallow layer feature; Step S2: performing featureextraction and refinement on the shallow layer feature by using aniterative up and down sampling method to obtain a plurality of outputfirst features and a plurality of output second features; Step S3:performing feature decoding on the plurality of output second features,and concatenating decoded features along channel dimension to obtainfeatures after a plurality of concatenation; and Step S4: performingweight distribution on the features after the plurality of concatenationto obtain final features, and restoring an image by using the finalfeatures.
 2. The neural network-based high-resolution image restorationmethod according to claim 1, wherein during feature extraction of thetarget frame in the network input, feature extraction is performed onthe target frame by using one or two convolutional layers to obtain thefirst feature; and during feature extraction of the first frame and anadjacent frame of the first frame and the optical flow between the firstframe and the adjacent frame, feature extraction is performed on thefirst frame, a dense optical flow between the first frame and a secondframe, and the second frame in a low-resolution image sequence by usingone or two convolutional layers.
 3. The neural network-basedhigh-resolution image restoration method according to claim 1, wherein adetermination method of performing feature extraction and refinement onthe shallow layer feature by using an iterative up and down samplingmethod comprises: determining whether there are uncalculated adjacentframes in the shallow layer feature, wherein if yes, featureconcatenation is performed on one obtained output first feature and afeature of an optical flow between a next frame and an adjacent frame ofthe next frame for use as an input for next iteration, and cycliciteration is performed until all input frames have been calculated; orif not, the process enters Step S3.
 4. The neural network-basedhigh-resolution image restoration method according to claim 1, whereinduring the iterative up and down sampling process, a process of a singleiterative up and down sampling comprises: a first convolutional layer, afirst deconvolutional layer, a second convolutional layer, a seconddeconvolutional layer, a third convolutional layer, and a thirddeconvolutional layer.
 5. The neural network-based high-resolution imagerestoration method according to claim 4, wherein the first convolutionallayer and the first deconvolutional layer use the same convolutionkernel, step size, and channel quantity; the second convolutional layerand the second deconvolutional layer use the same convolution kernel,step size, and channel quantity; and the third convolutional layer andthe third deconvolutional layer use the same convolution kernel, stepsize, and channel quantity.
 6. The neural network-based high-resolutionimage restoration method according to claim 4, wherein an input of thefirst convolutional layer is the shallow layer feature, an input of thefirst deconvolutional layer is a result of the first convolutionallayer, an input of the second convolutional layer is a differencebetween a result of the first deconvolutional layer and the shallowlayer feature, an input of the second deconvolutional layer is a resultof the second convolutional layer, an input of the third convolutionallayer is a result of the second deconvolutional layer, and an input ofthe third deconvolutional layer is a difference between a result of thethird convolutional layer and the result of the second deconvolutionallayer.
 7. The neural network-based high-resolution image restorationmethod according to claim 1, wherein the number of iterative up and downsampling process is adjusted according to a requirement of a networkscale.
 8. The neural network-based high-resolution image restorationmethod according to claim 3, wherein during the iterative up and downsampling process, an output second feature obtained from each iterationis saved.
 9. The neural network-based high-resolution image restorationmethod according to claim 1, wherein during restoration of the image byusing the final features, one or two convolutional layers are used. 10.A neural network-based high-resolution image restoration system,comprising: a feature extraction module, configured to: perform featureextraction on a target frame in a network input to obtain a firstfeature, perform feature extraction on a first frame and an adjacentframe of the first frame and an optical flow between the first frame andthe adjacent frame to obtain a second feature, and concatenate the firstfeature and the second feature to obtain a shallow layer feature; anencoding and decoding module, configured to perform feature extractionand refinement on the shallow layer feature by using an iterative up anddown sampling method to obtain a plurality of output first features anda plurality of output second features; an encoding module, configuredto: perform feature decoding on the plurality of output second features,and concatenate decoded features along channel dimensionality to obtainfeatures after a plurality of concatenation; and a weight distributionmodule and a restoration module, configured to: perform weightdistribution on the features after the plurality of concatenation toobtain final features, and restore an image by using the final features.